foundation model

A foundation model is anymodel that is trained on broad data (generally using self-supervision at scale) that can be adapted(e.g., fine-tuned) to a wide range of downstream tasks; current examples include BERT [Devlin et al .2019], GPT-3 [Brown et al . 2020], and CLIP [Radford et al . 2021].

On the Opportunities and Risks of Foundation ModelsOn the Opportunities and Risks of Foundation Models 2023

Many of these models are described as foundation models. This term, coined in 2021 by researchers at the Stanford Institute for Human-Centered Artificial Intelligence, broadly describes AI models that acquire the capacity to perform well across a range of different problems through a process of ‘pretraining’ on enormous unannotated datasets.

Foundation models build on ChatGPT tech to learn the fundamental language of biology - Nature Biotechnology Michael Eisenstein 2024

Wang is one of a growing number of scientists who see ‘foundation models’ as a powerful solution. The term describes AI algorithms that go through a pre-training process in which they are fed tremendous amounts of unlabelled data.

Self-driving laboratories, advanced immunotherapies and five more technologies to watch in 2025 Michael Eisenstein 2025